Time-varying Linear Regression with Total Variation Regularization

We consider modeling time series data with time-varying linear regression, a model that allows the weight matrix to vary at every time point but penalizes this variation with the (multivariate) total variation norm. This corresponds to simultaneously learning the parameters of multiple linear systems as well as the change points that describe when the underlying process switches between linear models. Computationally, this formulation is appealing as parameter estimation can be done using convex methods; we derive a fast Newton-like algorithm by considering the dual problem and by exploiting sparsity with an active set approach. We also develop an extension for prediction using the learned parameters with a kernel density estimator that exploits recurrent behavior in the time series. Our motivating example is the problem of modeling and predicting energy consumption—specifically learning models of home appliances which tend to be well-described as switched linear systems. On synthetic data we demonstrate that our algorithm is significantly faster than a straightforward implementation using ADMM; however, we also observe that the total variation norm often over-segments suggesting that some applications may require an additional polishing step. On real data we show that our method learns a sparse set of parameters describing the energy consumption of a refrigerator and enables us to predict future consumption significantly better than standard methods.